.\" SPDX-License-Identifier: BSD-2-Clause
.\" Copyright (C) 2019 - 2021 Intel Corporation.
.\"
.TH "MEMKINDALLOCATOR" 3 "2019-09-24" "Intel Corporation" "MEMKINDALLOCATOR" \" -*- nroff -*-
.SH "NAME"
libmemkind::static_kind::allocator<T> \- The C++ allocator compatible with the C++ standard library allocator concepts
.br
.BR Note:
.I memkind_allocator.h
functionality is considered as stable API (STANDARD API).
.SH "SYNOPSIS"
.nf
.B #include <memkind_allocator.h>
.sp
.B Link with -lmemkind
.sp
.BI "libmemkind::static_kind::allocator(libmemkind::kinds " "kind" );
.br
.BI "template <typename U> libmemkind::static_kind::allocator<T>::allocator(const libmemkind::static_kind::allocator<U>&)" " "noexcept;
.br
.BI "template <typename U> libmemkind::static_kind::allocator(const allocator<U>&& " "other" ) " "noexcept;
.br
.BI "libmemkind::static_kind::allocator<T>::~allocator();
.br
.BI "T *libmemkind::static_kind::allocator<T>::allocate(std::size_t " "n" ) " "const;
.br
.BI "void libmemkind::static_kind::allocator<T>::deallocate(T " "*p" ", std::size_t " "n" ) " "const;
.br
.BI "template <class U, class... Args> void libmemkind::static_kind::allocator<T>::construct(U " "*p" ", Args... " "args" ) " "const;
.br
.BI "void libmemkind::static_kind::allocator<T>::destroy(T " "*p" ) " "const;
.fi
.SH "DESCRIPTION"
The
.BR libmemkind::static_kind::allocator<T>
is intended to be used with STL containers to allocate from static kinds memory. Memory management is based on memkind library. Refer
.BR memkind (3)
man page for more details.
.PP
The
.BR libmemkind::kinds
specifies allocator static kinds of memory, representing type of memory which offers different characteristics. The available types of allocator kinds of memory:
.PP
.B libmemkind::kinds::DEFAULT
Default allocation using standard memory and default page size.
.PP
.B libmemkind::kinds::HIGHEST_CAPACITY
Allocate from a NUMA node(s) that has the highest capacity among all nodes in the system.
.PP
.B libmemkind::kinds::HIGHEST_CAPACITY_PREFERRED
Same as
.B libmemkind::kinds::HIGHEST_CAPACITY
except that if there is not enough memory in the NUMA node that has the highest capacity in the local domain
to satisfy the request, the allocation will fall back on other memory NUMA nodes.
.BR Note:
For this kind, the allocation will not succeed if there are two or more NUMA nodes
that have the highest capacity.
.PP
.B libmemkind::kinds::HIGHEST_CAPACITY_LOCAL
Allocate from a NUMA node that has the highest capacity among all NUMA Nodes from the local domain.
NUMA Nodes have the same local domain for a set of CPUs associated with them, e.g. socket or sub-NUMA cluster.
.BR Note:
If there are multiple NUMA nodes in the same local domain that have the highest
capacity - allocation will be done from NUMA node with worse latency attribute.
This kind requires locality information described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HIGHEST_CAPACITY_LOCAL_PREFERRED
Same as
.B libmemkind::kinds::HIGHEST_CAPACITY_LOCAL
except that if there is not enough memory in the NUMA node that has the highest capacity
to satisfy the request, the allocation will fall back on other memory NUMA nodes.
.PP
.B libmemkind::kinds::LOWEST_LATENCY_LOCAL
Allocate from a NUMA node that has the lowest latency among all NUMA Nodes from the local domain.
NUMA Nodes have the same local domain for a set of CPUs associated with them, e.g. socket or sub-NUMA cluster.
.BR Note:
If there are multiple NUMA nodes in the same local domain that have the lowest
latency - allocation will be done from NUMA node with smaller memory capacity.
This kind requires locality and memory performance characteristics information described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::LOWEST_LATENCY_LOCAL_PREFERRED
Same as
.B libmemkind::kinds::LOWEST_LATENCY_LOCAL
except that if there is not enough memory in the NUMA node that has the lowest latency
to satisfy the request, the allocation will fall back on other memory NUMA nodes.
.PP
.B libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL
Allocate from a NUMA node that has the highest bandwidth among all NUMA Nodes from the local domain.
NUMA Nodes have the same local domain for a set of CPUs associated with them, e.g. socket or sub-NUMA cluster.
.BR Note:
If there are multiple NUMA nodes in the same local domain that have the highest
bandwidth - allocation will be done from NUMA node with smaller memory capacity.
This kind requires locality and memory performance characteristics information described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL_PREFERRED
Same as
.B libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL
except that if there is not enough memory in the NUMA node that has the highest bandwidth
to satisfy the request, the allocation will fall back on other memory NUMA nodes.
.PP
.B libmemkind::kinds::HUGETLB
Allocate from standard memory using huge pages. Note: This kind requires huge pages configuration described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::INTERLEAVE
Allocate pages interleaved across all NUMA nodes with transparent huge pages disabled.
.PP
.B libmemkind::kinds::HBW
Allocate from the closest high bandwidth memory NUMA node at the time of allocation. If there is not enough high bandwidth memory to satisfy the request, errno is set to ENOMEM and the allocated pointer is set to
.IR "NULL" .
Note: This kind requires memory performance characteristics information described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HBW_ALL
Same as
.B libmemkind::kinds::HBW
except decision regarding closest NUMA node is postponed until the time of first write.
.PP
.B libmemkind::kinds::HBW_HUGETLB
Same as
.B libmemkind::kinds::HBW
except the allocation is backed by huge pages. Note: This kind requires huge pages configuration described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HBW_ALL_HUGETLB
Combination of
.B libmemkind::kinds::HBW_ALL
and
.B libmemkind::kinds::HBW_HUGETLB
properties. Note: This kind requires huge pages configuration described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HBW_PREFERRED
Same as
.B libmemkind::kinds::HBW
except that if there is not enough high bandwidth memory to satisfy the request, the allocation will fall back on standard memory.
.PP
.B libmemkind::kinds::HBW_PREFERRED_HUGETLB
Same as
.B libmemkind::kinds::HBW_PREFERRED
except the allocation is backed by huge pages. Note: This kind requires huge pages configuration described in
.B SYSTEM CONFIGURATION
section.
.PP
.B libmemkind::kinds::HBW_INTERLEAVE
Same as
.B libmemkind::kinds::HBW
except that the pages that support the allocation are interleaved across all high bandwidth nodes and transparent huge pages are disabled.
.PP
.B libmemkind::kinds::REGULAR
Allocate from regular memory using the default page size. Regular means general purpose memory from the NUMA nodes containing CPUs.
.PP
.B libmemkind::kinds::DAX_KMEM
Allocate from the closest persistent memory NUMA node at the time of allocation. If there is not enough memory in the closest persistent memory NUMA node to satisfy the request,
.I errno
is set to
.B ENOMEM
and the allocated pointer is set to
.IR "NULL" .
.PP
.B libmemkind::kinds::DAX_KMEM_ALL
Allocate from the closest persistent memory NUMA node available at the time of allocation. If there is not enough memory on any of persistent memory NUMA nodes to satisfy the request,
.I errno
is set to
.B ENOMEM
and the allocated pointer is set to
.IR "NULL" .
.PP
.B libmemkind::kinds::DAX_KMEM_PREFERRED
Same as
.B libmemkind::kinds::DAX_KMEM
except that if there is not enough memory in the closest persistent memory NUMA node to satisfy the request, the allocation will fall back on other memory NUMA nodes.
.BR Note:
For this kind, the allocation will not succeed if two or more
persistent memory NUMA nodes are in the same shortest distance to the same CPU on which process is eligible to run.
Check on that eligibility is done upon starting the application.
.PP
.B libmemkind::kinds::DAX_KMEM_INTERLEAVE
Same as
.B libmemkind::kinds::DAX_KMEM
except that the pages that support the allocation are interleaved across all persistent memory NUMA nodes.
.PP
All public member types and functions correspond to standard library allocator concepts and definitions. The current implementation supports C++11 standard.
.PP
Template arguments:
.br
.I T
is an object type aliased by value_type.
.br
.I U
is an object type.
.PP
.BR Note:
.br
.BI "T *libmemkind::static_kind::allocator<T>::allocate(std::size_t " "n")
allocates memory using
.BR memkind_malloc ().
Throw
.I std::bad_alloc
when:
.RS
.I n
= 0
.RE
.br
.RS
.br
or there is not enough memory to satisfy the request.
.RE
.PP
.BI "libmemkind::static_kind::allocator<T>::deallocate(T " "*p" ", std::size_t " "n")
deallocates memory associated with pointer returned by
.BR allocate ()
using
.BR memkind_free ().
.SH "SYSTEM CONFIGURATION"
Interfaces for obtaining 2MB (HUGETLB) memory need allocated
huge pages in the kernel's huge page pool.
.TP
.B HUGETLB (huge pages)
Current number of "persistent" huge pages can be read from
.I /proc/sys/vm/nr_hugepages
file.
Proposed way of setting hugepages is:
.BR "sudo sysctl vm.nr_hugepages=<number_of_hugepages>" .
More information can be found here:
.UR https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt
.UE
.PP
Interfaces for obtaining locality information are provided by
.I libhwloc
dependency.
Functionality based on locality requires that memkind library is configured and built with the support of
.I libhwloc
(./configure --enable-hwloc).
.PP
Interfaces for obtaining memory performance characteristics information are based on
.I HMAT
(Heterogeneous Memory Attribute Table)
.UR https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf
.UE
Functionality based on memory performance characteristics requires that platform configuration
fully supports HMAT and memkind library is configured and built with the support of
.I libhwloc
(./configure --enable-hwloc).
.PP
.BR Note:
For a given target NUMA Node, the OS exposes only the performance characteristics of the best performing NUMA node.
.PP
.I libhwloc
can be reached on:
.UR https://www.open-mpi.org/projects/hwloc
.UE
.SH "COPYRIGHT"
Copyright (C) 2019 - 2021 Intel Corporation. All rights reserved.
.SH "SEE ALSO"
.BR memkind (3)
